The market place is a collection of scripts in various languages.
Data Cleaning - replace nulls with the mean
Replace the nulls of a column with the mean of that column
- Input a numeric column with nulls
- Output a numeric column with means instead of nulls
- The script can be used for cleaning nulls (by replacing them with the mean)
Data Cleaning - replace nulls with the median
Replace the nulls of a column with the median of that column
- Input a numeric column with nulls
- Output a numeric column with median instead of nulls
- The script can be used for cleaning nulls (by replacing them with the median)
Data Cleaning - replace nulls with zero
Replace the nulls of a column with zeros
- Input a numeric column with nulls
- Output a numeric column with zeros instead of nulls
- The script can be used for cleaning nulls (by replacing them with zeros)
Replace Empty String
Replace empty strings with 'Not Found'
- Input a nominal column
- Output a column with empty strings replaced with 'Not Found' (or any other string specified in the script)
Round Numbers
Round numbers in the given column to the nearest integer
- Input a numeric column
-
Output a numeric column with the numbers rounded to a specified precision (2.12345 with precision 2 is 2.12)
Upper case
Modify a nominal column to be in its upper case form
- Input a nominal column
- Output the column in upper case
Lower case
Modify a nominal column to be in its lower case form
- Input a nominal column
- Output the column in lower case
Date differences
Get the differences between dates in successive rows
- Input a date column
- Output column of differences between dates
Round numbers in strings
Finds and rounds all the numbers in a nominal column
- Input nominal column
-
Output a nominal column with the numbers rounded to a specified precision (2.12345 with precision 2 is 2.12)
Remove rows with missing data
Remove rows with missing data
- Input all columns
- Output all columns, excluding rows with missing data
Sentiment analysis
Estimates for each line of text if it’s a positive or negative phrase by counting the positive and negative words (dictionaries are download from pyramid website)
- Input a text column where each line has more than one word
- Output a categorical column with positive / negative categories.
- This script can be used for analyzing restaurant reviews/ book review etc.
Create outlier annotation
Analyzing numeric data in determine if a value is an outlier
Note- the sensitivity of the outlier can be adjusted by changing outlier Upper Threshold, increase its value will produce more outlier and visa-versa.
- Input a numeric column
- Output a categorical column with outlier / not-outlier categories.
- The script can be used for coloring outliers
Create outlier annotation by standard deviation
If an outlier is detected and confirmed by standard deviation, this function creates a column with an is-outlier annotation.
Note that the sensitivity of the outlier can be adjusted by changing standard deviation Number. Increasing its value will produce more outliers, while reducing it will produce fewer outliers.
- Input a numeric column
- Output a categorical column with outlier/not-outlier categories.
- The script can be used for coloring outliers
Remove the values most different from the mean
Analyzing numeric data and determine if a value is an outlier, if so replace it with the mean
- Input a numeric column
- Output numeric column with clean values
- The script can be used for using a cleaner data.
Clean text
Remove all non-Alphanumeric characters
- Input the text column to be cleaned
- Output the column without the non-Alphanumeric characters
- The script can only be used for health data for infants
Matrix transpose
Transpose matrix- rows to columns- columns to rows
- Input a matrix nXm the following implementation is 3X3 but can be adjusted
- Output the transpose of the matrix
Difference matrix
Return a numeric matrix with the difference between rows (instead of the actual values).
- Input numeric columns
- Output a numeric matrix with the differences between rows
Mack Chain-Ladder
The chain-ladder method is a prominent actuarial loss reserving technique.
The chain-ladder method is used in both the property and casualty and health insurance fields.
- Input numeric accident time (period), claim time (period), and losses.
- Output is a Chain ladder table
- The script can be used for an insurance data set for estimating future claims.
Clark LDF method
Analyze loss triangle using Clark's LDF (loss development factor) method.
- Input three numeric accident time (period), claim time (period) and losses.
- Output the Clark LDF estimation
- The script can be used for insurance dataset for estimating future claims.
Geo Distance
Calculates the distance between two geographical locations of each data row.
- Input two locations represented by longitude and latitude columns
- Output the distance between the two locations
Geo Distance from Central Location
Calculates the distance between a geographical location of each data row and a central location.
- Input a location represented by longitude and latitude
- Output the distance between the location and the central location that is hard coded in the script
Moving Average
Calculates the moving average of the last N rows.
- Input a numeric column and a column to use for sorting (i.e. date)
- Output a column with the moving average
Get stock daily data
Download data from Yahoo website, change stock name, start date and end date as needed
- Input a column with the names of the stocks
- Output the data from Yahoo for each of the stocks
Country from Address
Gets the country to which an address (full or partial) belongs.
- Input a column of addresses
- Output a column of the countries of the addresses
- The script can be used when the address is available but not the country of the address
Address from Coordinates
Gets the address that matches the latitude and longitude (coordinates) of the data entries.
- Input a column of coordinates (or two columns, one of longitudes and one of latitudes)
- Output a column of the matching addresses
Coordinates from Address
Gets coordinates the addresses
- Input a column of addresses
- Output the coordinates of each address
Get stocks data
Get stock exchange data for multiple stocks over a period of time using Google Finance API
-
Input a hardcoded array of the stock ids, or a nominal column with the stock IDs
- Output a table with stock exchange data
- Downloads googlefinance.client, pandas
Sharp Ratio
Calculates the sharp ratio for different stocks from a specified date
- Input a column of stock names
- Output the sharp ratio of each stock
Basket analysis
Basket analysis for estimating a group of products that will indicate the next group of products to be purchased.
Note- it is required to change the 1st code line according to the input, adding each product to the data frame.
- Input 2 columns: 1. Transaction number (ID), 2. Product Name
- Output- rules, confidence, lift and support table, with the name of the products in each cell.
-
Input N(unknown) Boolean matrix , rows should indicate a transaction, with each column representing a different product. The matrix is filled with false/true values indicating if a product was purchased at the transaction
-
Output- rules table with the name of the products in each cell.
-
The script can be used for analyzing a group of products.
Risk return ratio
Calculates the return / risk ratio. Basically the Sharpe ratio without factoring in the risk-free rate.
- Input a vector with stock name
- Output a new table with a stock name and its risk return ratio
- The script is importing data from yahoo and the start date & end date can be adjusted.
Birch clustering
Implements the Birch clustering algorithm.
- Inputs are: numeric columns (the default are 3 but can be modified)
- Output a new vector with cluster number
- Hierarchical clustering for large dataset
Record expander by month
Date transformation from range to column: start date-end date columns to a monthly column.
- Inputs are: Start Date- date, end date- date, Param1- the parameter to expend
- Output a new table with a line for each month between the date pairs
- General expander by month (for future analysis)
Record expander by days
Date transformation from range to column: start date-end date columns to a daily column.
- Inputs are: Start Date- date, end date- date, Param1- the parameter to expend
- Output a new table with a line for each day between the date pairs
- General expander by day (for future analysis)
App is on-counter
Counts the number of time an application was opened.
- Input a Boolean array
- Output a new column with a switch counter
- Can be used for an application provider that is required to count the number of times it turned on
Service-counter
Counts the number of times the service was requested.
- Input a Boolean array
- Output a new column with a switch counter
- Can be used for a service provider that is required to count the number of time it switched to true
Month counter
Counts the number of months and produces a count table.
- Input date-time column
- Output a new table with a max of 12 rows and for each month its counter
- The script can be used for any date-time input where a group by month is needed
Weekday counter
Counts the number of weekday and produces a count table.
- Input date-time column
- Output a new table with a max of 7 rows and for each weekday its counter
- The script can be used for any date-time input where a group by weekday is needed
Break-even quantity
Unit Contribution of the new product/Unit Contribution of the old product.
- Input three numeric arrays: fixed costs, average Price Per Unit and average Cost Per Unit
- Output numeric column of the ratio
- The script can be used for analyzing the BECR
Break-even cannibalization rate
Unit Contribution of the new product/Unit Contribution of the old product.
- Input two numeric arrays: unit contribution of new product and unit contribution of old product
- Output numeric column of the ratio
- The script can be used for analyzing the BECR
Estimate blood pressure in infants at birth based on body weight
A simple estimation of blood pressure by body weight.
- Input a numeric column of body weight
- Output numeric column of blood pressure
- The script can be only for health data for infants.
Estimate red cell volume in infants based on body weight
A simple estimation of red cell volume by body weight.
- Input a numeric column of body weight
- Output numeric column of red cell volume
- The script can be only for health data for infants.